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COLOR SENSOR 

STATEMENT OF GOVERNMENT INTEREST 
The invention described herein may be manufactured by or for 
the Government of the United States of America for Governmental 
purposes without the payment of any royalties thereon or 
therefore. 

CROSS-REFERENCE TO RELATED APPLICATIONS 
This patent application is co-pending with related patent 
applications entitled NEURAL DIRECTORS (U.S. Patent Application 
Ser. No. 09/436,957), NEURAL SENSORS (U.S. Patent Application 
Ser. No. 09/436,956), STATIC MEMORY PROCESSOR (U.S. Patent 
Application Ser. No. 09/477,638), DYNAMIC MEMORY PROCESSOR (U.S. 
Patent Application Ser. No. 09/477,653), MULTIMODE INVARIANT 
PROCESSOR (U.S. Patent Application Ser. No. 09/641,395) and A 
SPATIAL IMAGE PROCESSOR (A trLuxn fey Docket No. 7 -73 4 6) , by the same 
inventor as this patent application. 

BACKGROUND OF THE INVENTION 
(1) Field of the Invention 

The invention relates generally to the field of color 
sensors and more particularly to color sensors having neural 
networks with a plurality of hidden layers, or multi- layer neural 
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networks, and further to a new neural network processor for 




2 


sensing color in optical image data. 
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(2) Description of the Prior Art 
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Electronic neural networks have been developed to rapidly 
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identify patterns in certain types of input data, or accurately 
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to classify the input patterns into one of a plurality of 
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predetermined classifications. For example, neural networks have 
been developed which can recognize and identify patterns, such as 
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the identification of hand-written alphanumeric characters, in 
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response to input data constituting the pattern of on and off 
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picture elements, or "pixels", representing the images of the 
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characters to be identified. In such a neural network, the pixel 
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pattern is represented by, for example, electrical signals 
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coupled to a plurality of input terminals, which, in turn, are 
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connected to a number of processing nodes, each of which is 
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associated with one of the alphanumeric characters which the 
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neural network can identify. The input signals from the input 
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terminals are coupled to the processing nodes through certain 
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weighting functions, and each processing node generates an output 
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signal which represents a value that is a non- linear function of 
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the pattern of weighted input signals applied thereto. Based on 
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input terminals, if the input signals represent a character that 
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can be identified by the neural network, the one of the 




25 


processing nodes associated with that character will generate a 
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1 positive output signal, and the others will not. On the other 

2 hand, if the input signals do not represent a character that can 

3 be identified by the neural network, none of the processing nodes 

4 will generate a positive output signal. Neural networks have 

5 been developed which can perform similar pattern recognition in a 

6 number of diverse areas . 

7 The particular patterns that the neural network can identify 

8 depend on the weighting functions and the particular connections 

9 of the input terminals to the processing nodes. The weighting 

10 functions in, for example, the above -described character 

11 recognition neural network, essentially will represent the pixel 

12 patterns that define each particular character. Typically, each 

13 processing node will perform a summation operation in connection 

14 with values representing the weighted input signals provided 

15 thereto, to generate a sum that represents the likelihood that 

16 the character to be identified is the character associated with 

17 that processing node. The processing node then applies the non- 
18 linear function to that sum to generate a positive output signal 

19 if the sum is, for example, above a predetermined threshold 

20 value. Conventional non-linear functions which processing nodes 

21 may use in connection with the sum of weighted input signals is 

22 generally a step function, a threshold function, or a sigmoid, in 

23 all cases the output signal from the processing node will 

24 approach the same positive output signal asymptotically. 
25 



-3- 





1 




2 




3 




4 




5 




6 




7 
8 


p 


y 
10 


•.fi 




i K z 




? f : 


12 


ip 




iU 


13 


|# 


14 








15 




16 








17 




18 




19 




20 




21 




22 




23 




24 




25 



Before a neural network can be useful, the weighting 
functions for each of the respective input signals must be 
established. In some cases, the weighting functions can be 
established a priori. Normally, however, a neural network goes 
through a training phase, in which input signals representing a 
number of training patterns for the types of items to be 
classified, for example, the pixel patterns of the various hand- 
written characters in the character- recognition example, are 
applied to the input terminals, and the output signals from the 
processing nodes are tested. Based on the pattern of output 
signals from the processing nodes for each training example, the 
weighting functions are adjusted over a number of trials. After 
the neural network has been trained, during an operational phase 
it can generally accurately recognize patterns, with the degree 
of success based in part on the number of training patterns 
applied to the neural network during the training stage, and the 
degree of dissimilarity between patterns to be identified. Such 
a neural network can also typically identify patterns that are 
similar, but not necessarily identical, to the training patterns. 

One of the problems with conventional neural network 
architectures as described above is that the training 



addition, under the back-propagation method, the neural network 
25 may result in erroneous results that may require restarting of 
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training. Even after a neural network has been through a 
training phase, confidence that the best training has been 
accomplished may sometimes be poor. If a new classification is 
to be added to a trained neural network, the complete neural 
network must be retrained. In addition, the weighting functions 
generated during the training phase often cannot be interpreted 
in ways that readily provide understanding of what they 
particularly represent. 

Edwin H. Land's Retinex theory of color vision is based upon 
"three color" experiments performed before 1959. A simple 
"mishap" showed that three colors were not always required to see 
accurate color. Land used a short and long record of brightness 
data (black and white transparencies) to produce color perceived 
by human eyes and not by photographic means . He demonstrated a 
15 perception of a full range of pastel colors using two very 

similar in color light sources such as yellow, at 579 nm and 
17 yellow orange, at 599 nm ("Experiments in Color Vision", Edwin H. 
Land, Scientific American, Vol. 200 No. 5, May 1959) . Land found 
that in some two record experiments all colors present were not 
perceived. Although Land demonstrated that two records provided 
color perceptions, he constructed his Retinex theory upon three 
records such as his long, medium and short records (An 
Alternative Technique for the Computation of the Designator in 
the Retinex Theory of Color Vision", Edwin H. Land, Proceedings 
of the National Academy of Sciences, Vol. 83, 1986). The 
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1 invention herein is related to human color perception discovered 

2 during Land's color vision experiments as reported in 1959. 

3 The "Trichromatic" theory in human color vision has been 

4 accepted on and off since the time of Thomas Young in 1802 (A 

5 Vision in the Brain", S. Zeki, Blackwell Scientific Publishing, 

6 1993) . Still and video electronic camera designs are correctly 

7 based upon the trichromatic theory but the current designs are 

8 highly subjective to color error reproduction due to changes in 

9 the ambient light color temperatures and color filtrations. The 

13 10 device in this invention senses color using a new "bichromatic" 

in 

ffl 11 theory, which includes a mechanism that insures color constancy 

■y 12 over a large range of ambient color temperatures. The use of two 

i : : 
' ?~ 

ilJ 13 lightness records as used by Land in 1959 is one key to this 

14 invention. 

15 The bichromatic theory is based upon an interpretation of a 

16 biological color process that occurs in the eyes and brain of 

17 humans and in some animals. The bichromatic theory is defined as 

18 a system that functions together under the following assumptions, 

19 accepted principles and rules of procedure, for which FIGS. 4A 

20 and 4B are provided for support : 

21 (1) The system is a color sensing retina. There are at 

22 least two photo transducers in each pixel space in the retina, 

23 shown in FIG. 4B as TR(HI) and TR(LO) . 

24 (2) The two photo transducers sense the color of the light 

25 at each pixel's position in a scene of color focused on the 
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1 retina. Each of the at least photo transducers contains a 

2 different spectral response and the wavelength difference between 

3 the peaks of a pair of these responses is called the waveband or 

4 the spectral bandwidth of the two photo transducers . 

5 (3) The two photo transducers have overlapping spectral 

6 logarithmic responses where their slopes are opposing each other 

7 as indicated in FIG. 4A. 

8 (4) The photo transducers have at least two controlled gain 

9 amplifiers (CGA) and at least two common controlling circuits. 

10 There is one controlled gain amplifier for each photo transducer 

11 where each of the at least two common controlling circuits 

12 controls the controlled gain amplifiers for all the photo 

13 transducers of the same spectral response. 

14 (5) The highest energy value in the retina, or the peak 

15 energy from a photo transducer of a specific spectral response, 

16 controls the output of the common controlling circuits that 

17 normalize the logarithmic response of all photo transducers with 

18 the same spectral response. Thus, it is always the peak energy 

19 photo transducer no matter its position in the retina that 

20 controls the common mode gain. The peak response of a photo 

21 transducer is relative to the best matched wavelength of energy 

22 for all wavelengths of light impinging on the color retina. 

23 Therefore, each photo transducer will be continuously normalized 

24 to the peak photo transducer signal in response to changes in 

25 ambient lighting. 
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1 (6) In a general discussion herein a normalized photo 

2 transducer or a normalized pixel includes the controlled gain 

3 amplifier as part of its response. A photo transducer sensing 

4 the peak energy or a peak energy sensing photo transducer will 

5 only be called as such thus a normalized photo transducer will 

6 not specifically include a peak energy sensing photo transducer. 

7 (7) There are three color coordinates called hue, lightness 

8 and saturation. Three degrees of freedom are required to 

9 categorize all combinations of color attributes. Two points in a 

10 two dimensional space can be connected by a line. Combinations 

11 of positions of these two points in space can produce at least 

12 three families of lines in the two dimensional space. The line 

13 families are horizontal, vertical and sloped. FIG. 4A shows a 

14 two dimensional graph of the responses of two normalized photo 

15 transducers. A straight line on the graph may represent the two 

16 output values of the normalized photo transducers for a specific 

17 input light condition. The graph coordinates are light 

18 wavelength for the horizontal axis and signal in a natural log 

19 scale for the vertical axis. Output values of the two normalized 

20 photo transducers can be represented by three families of lines. 

21 (8) The response "curve" of a normalized photo transducer 

22 output signal for a normalized light energy input is shown as a 

23 straight line, from the maximum response at its wavelength, down 

24 to the bottom at the opposite side of the graph. Each response 

25 curve of the normalized photo transducers has opposing slopes 
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1 that cross each other. A normalized photo transducer response 

2 over the waveband is given as TR(b)=ce''", where: x equals the 

3 wavelength position in the normalized waveband relative to the 

4 maximum response of the photo transducer, i.e., (0 to 1); c, the 

5 conversion constant, equals one for a normalized light energy, 

6 or, alternately, an integrated CGA value; k equals approximately 

7 10; and b is the high or low transducer. The output signal level 

8 is symbolized by El for the low wavelength normalized photo 

9 transducer and E2 for the other. 

10 (9) A broad constant energy spectrum of visible light 

11 relative to its color temperature "flattens" its spectral energy 

12 curve as the color temperature increases from a deep red at 1000 

13 °K to a "slightly bluish" white at 10,000 °K. Thus, when the 

14 peak energy photo transducers normalize the retina's response, 

15 the results are equivalent to "whitening" the pixel's responses 

16 in the waveband of sensible colors. In other words, possibly 

17 different energies near the wavelength of the maximum 

18 sensitivities of the peak energy transducers contain 

19 approximately equal spectral energies at the output of the 

20 respective controlled gain amplifiers. This process develops a 

21 color constancy in ambient lights of different color 

22 temperatures . 

23 (10) A family of horizontal lines can represent the 

24 normalized photo transducer responses to a broadband family of 

25 white light from bright through gray to dark. Example 1 on the 
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graph is a representation of this family. A family of vertical 
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lines can represent a family of wavelengths in the waveband. 
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Example 2 on the graph is a representation of the wavelength of a 
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monochromatic light source. Families of sloped lines, from a 
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horizontal position to a vertical position, closely represent a 
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morphing from "white" to a monochromatic light. A change from 




7 
8 


white light to a light of a pure color is along the axis for the 
color attribute of saturation. Example 3 on the graph is a 




9 


representation of a pastel color. The three families of lines 




10 


are closely mapped to the three color coordinates of hue. 
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lightness and saturation, but not with an exact one to one 
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correlation. A combination of either set of three dimensions of 
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color attributes can be mapped into the other. The two response 
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values of a normalized pixel can represent a line that can move 




15 


in combinations of the three coordinate ways to represent exact 
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changes in lightness, hue and saturation of colors. 
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(11) The output values of a normalized pixel, in response 
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to a monochromatic light, shall exhibit proportional photo 




19 


transducer output values of El and E2 that are relative to the 




20 


wave length of the light in the waveband between the two photo 




21 


transducers. In the case where there is a broad spectrum of 




22 


light illuminating an object, the different reflective bands of 




23 


light relative to the wavelength responses of the normalized 




24 
25 


pixel will produce photo transducer output values in proportional 
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1 to values that would be generated by a colored light of the 

2 perceived color. 

3 (12) Changing the pixel's response from straight lines to 

4 curved lines on the logarithmic scale does not change the two 

5 point families of lines but it will change the form of the 

6 mapping between the two different color attributes. 

7 (13) There is another control mode that increases the 

8 dynamic range of the sensibility to light of all photo 

9 transducers in the retina. This control sums the energy of all 

10 spectral responses to adjust an iris to maintain a constant 

11 energy to the retina under varying environmental lighting 

12 intensities. 

13 (14) This bichromatic theory projects that human color 

14 vision may not be as commonly believed. The human retina ^ 

15 contains three color cones to sense three different wavelengths 

16 of light, which may be used as two color pairs such as a 

17 blue-green pair and a red-green pair. Each color pair is 

18 processed in the visual cortex to map colors that can be 

19 associated to the visual space of an object in a scene. The two 

20 color pairs and processing will produce a wide range of colors 

21 sensed and a wide range of color constancy. Edwin H. Land's 

22 pre- 1959 experiments using two black and white transparencies and 

23 two color filters produced a perception of color. The color 

24 perception and constancy occur because the brightest area of one 

25 of the projected transparencies normalizes the response of the 
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1 appropriate set of human color cones to the specific color 

2 projected and the same occurs for the other transparency. The 

3 normalized human retina now sees varying ratios of brightness 

4 (energy) over the visual scene, which produces the perception of 

5 colors of light for the specific color temperatures of natural or 

6 artificial light. The bichromatic theory of color is an 

7 integration of the above fourteen theorems that together define 

8 the workings of color perception and color constancy. 

9 

10 SUMMARY OF THE INVENTION 

11 It is therefore an object of the invention to provide a new 

12 and improved neural network color sensor. 

13 It is a further object to provide a neural network color 

14 sensor in which the weighting functions may be determined a 

15 priori. 

16 Another object of the present invention is to provide a 

17 neural network color sensor, which can be trained with a single 

18 application of an input data set. 

19 In brief summary, the color sensor generates color 

20 information defining colors of an image, comparison of colors 

21 illuminated under two or more light sources and boundaries 

22 between different colors . The color sensor includes an input 

23 section, a color processing section, a color comparison section, 

24 a color boundary processing section and a memory processing 

25 section. The input section includes an array of transducer 
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1 pairs, each transducer pair defining one of a plurality of pixels 

2 of the input section. Each transducer pair comprises at least 

3 two transducers, each generating an output having a peak at a 

4 selected color, the selected color differing as between the two 

5 transducers, and each transducer having an output profile 

6 comprising a selected function of color. The color processing 

7 section includes a plurality of color pixel processors, each 

8 receiving the outputs from the two transducers comprising the 

9 transducer pair associated with a pixel. In response, the color 

10 processing section generates a color feature vector 

11 representative of the brightness of the light incident on the 

12 pixel and a color value corresponding to the ratio of outputs 

13 from the transducers comprising the transducer pair associated 

14 with the pixel. The color boundary processing section generates 

15 a plurality of color boundary feature vectors, each associated 

16 with a pixel, each representing the difference between the color 

17 value generated by the pixel color processor for the respective 

18 pixel and color values generated by the pixel color processor for 

19 pixels neighboring the respective pixel. 

20 The color boundary sensor produces object shape feature 

21 vectors from a function of the differences in color. This color 

22 boundary sensor can sense a colored object shape in a color 

23 background where a black and white sensing retina could not 

24 detect differences in lightness between the background and the 

25 object. The color comparator processor can measure and compare 
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1 the reflective color of two objects, even when each object is 

2 illuminated by two lights of different color temperatures. 

3 Thememory processor section provides a process to recognize a 

4 color, a boundary of color and a comparison of colors. 

5 

6 BRIEF DESCRIPTION OF THE DRAWINGS 

7 A more complete understanding of the invention and many of 

8 the attendant advantages thereto will be readily appreciated as 

9 the same becomes better understood by reference to the following 

10 detailed description when considered in conjunction with the 

11 accompanying drawings wherein corresponding reference characters 

12 indicate corresponding parts throughout the several views of the 

/ 

13 drawings and ywherem: 

14 FIG. A. is a functional block diagram of a color sensor 

V 

15 constructed xrv accordance with the invention; 

16 FIG. lA yis an expanded view of a transducer pair; 

17 FIG. 2/is a functional block diagram of a color processor, 

18 which is useful in the color sensor depicted in FIG. 1; 

/ 

19 FIG. 3 is a functional block diagram of a color boundary 

20 processor, wliich is useful in the color sensor of in FIG. 1; 

/ 

21 FIG. 4A is an example of the responses of two normalized 

22 photo transducers used in the color sensor; and 

/ 

23 FIG. 4'B is a schematic illustration of the theorems defining 

24 the workings of the color sensor. 

25 
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1 DESCRIPTION OF THE PREFERRED EMBODIMENT 

2 FIG. 1 is a functional block diagram of a color sensor 10 

3 constructed in accordance with the invention. By way of 

4 background, the color sensor 10 operates in accordance with a 

5 "bi- chromatic" mechanism of color recognition, which is theorized 

6 as being similar to the way in which human beings see and 

7 recognize color. In the conventional "tri-chromatic" color 

8 recognition mechanism, any color light, either reflected or 

9 incidental, can be generated combining three different color 

10 illuminations. In the reverse, i.e., color recognition, any 

11 input color can be represented or analyzed as a combination of 

12 three colors, i.e., base colors. Accordingly three transducers, 

13 each sensing one of the base three colors, can be used to 

14 determine the contribution of each of the base colors in the 

15 input color. In the bi-chromatic mechanism, colors can be 

16 distinguished using two color transducers, which have peak 

17 sensitivity at different colors, and provide a known output 

18 signal response as a function of the input color. The color 

19 sensor 10 determines, for an input image, the distribution of 

20 colors over the image, using two color transducers to identify 

21 the color at each point (that is, for each pixel or picture 

22 element) in the image. The color boundary process produces 

23 object shape features relative to the boundaries between 

24 different colors. The color comparator process produces 

25 comparative features relative to a "true reflective color" in 
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1 ambient lights of different color temperatures. The reading of a 

2 "true reflective color" in an ambient light of a color 

3 temperature and the reading of the same "true reflective color" 

4 in an ambient light of a second color temperature is a process 

5 that mimics human color constancy. 

6 With reference to FIG. 1, the color sensor 10 includes an 

7 input section 11, a color processing section 12 and a color 

8 boundary processing section 13, a color comparison processor 19 

9 and a memory processor 29. The color processing section 12 and a 

10 color boundary processing section 13 both generate color and 

11 color boundary feature vectors, which may be provided to, for 

12 example, a memory processing section 14. The input section 11 

13 receives an image of an object and generates, for each point, or 

14 pixel, color information signals representative of the color at 

15 the particular point of the image. The input section 11 includes 

16 a "retina" 15, which comprises an array of transducer pairs 15(1) 

17 through 15(1X1) (generally identified by reference numeral 15 (m) 

18 and shown in the expanded view of FIG. lA) , which define the 

19 pixels of the image. Each transducer pair comprises two 

20 transducers, which have output peaks at two different frequencies 

21 and which provide a predetermined output value as a function of a 

22 color wave band. Preferably, all of the pixels will have one 

23 transducer 15 (m) (1) which has a peak output at one frequency 

24 identified as 1 and the second transducer 15 (m) (2) having a peak 

25 output at a second frequency identified as 2. The input section 
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1 11 further includes a lens 26, which focuses an image of the 

2 object onto the retina 15, and an iris 17, which controls the 

3 intensity of light incident on the retina 15. 

4 The color processing section 12 uses the color information 

5 signals from the input section to generate, for each pixel, a 

6 local color feature vector representative of the color of the 

7 pixel. The color processing section 12 consists of a color 

8 processor array 20 and a feature fusion network array 23. The 

9 structure and operation of the color processing section 12 will 

10 be described in detail below in connection with FIG. 2. 

11 Similarly, the color boundary processing section 13 generates, 

12 for each pixel, a local color gradient feature vector that 

13 represents the gradient of the color at the pixel. The structure 

14 and operation of the color boundary processing section 13 will be 

15 described in detail below in connection with FIG. 3. The memory 

16 processor 29 is as described in STATIC MEMORY PROCESSOR, U.S. 

17 Patent Application Ser. No. 09/477,638. The parallel memory 

18 processors 16 and 18 are as described for the memory processor of 

19 the MULTIMODE INVARIANT PROCESSOR (U.S. Patent Application Ser. 

20 No. 09/641,395). The multi-mode invariant image processor, 

21 without its input sensor, is used for both parallel memory 

22 processors 16 and 18 . The possible multiple outputs of the 

23 parallel memory processor 18 are the colored input object (s) 

24 classifications. The output vector array of the parallel memory 

25 processor 16 is a Positional King Of the Mountain (PKOM) array 
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1 mapped to the pixels 15 (m) in the retina, which becomes a map of 

2 color classifications of each pixel. It is noted that the PKOM 

3 array is a neural network array internal to the parallel memory 

4 processor 16 and the remaining neural circuits to the normal 

5 output of the MULTIMODE INVARIANT PROCESSOR are not used. The 

6 memory processor 29 is a static memory processor and provides an 

7 output classification as a degree of color comparison. 

8 The local color feature vectors and the local color gradient 

9 feature vectors generated for all of the pixels are processed by 

10 the processing section 14 to, for example, classify the image 

11 into one of a plurality of image classes. The processing section 



ill 

hi 12 14 may comprise any of a plurality of processing elements for 



13 processing the vectors generated by the color processors 12, 13 

14 and/or 19. 



|h 15 FIG. 2 is a functional block diagram of color processing 

'm 16 section 12 and 19 as used in the color sensor of FIG. 1. With 

1^' 17 reference to FIG. 2, the color processing section 12 includes a 

18 plurality of pixel color processors 20(1) through 20 (M), 

19 generally identified by reference numeral 20 (m) . For each color 

20 processor 20 (m), a corresponding feature fusion network 23 (m) of 

21 color processing section 12 includes corresponding feature fusion 

22 neural directors 35(1) through 35 (M) and Multi King Of the 

23 Mountain (MKOM) 36(1) through 36 (M), generally identified by 

24 reference numerals 35 (m) and 36 (m), respectively. The structures 

25 of all of the pixel color processors 20 (m) are similar, and so 
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FIG. 2 depicts the structure of only one pixel color processor 
and the corresponding feature fusion neural director 35 (m) and 
MKOM 36 (m). Each pixel color processor 20 (m) processes the 
outputs generated by one of the transducer pairs in the retina 
11. The color processing section 12 also includes a common 
control 21, which controls all of the pixel color processors 
20 (m) in parallel, controls the iris 17 and receives pixel data 
from each color processor 20 (m). 

Each pixel color processor 20 (m) includes controlled gain 
10 amplifier (CGA) circuits 30(m)(l), 30(m)(2), which receive the 
color amplitude signals generated by the respective transducers 
15(m)(l), 15 (m) (2). Each CGA circuit 3 0 (m) (1) , 30 (m) (2) 
generates an output adjusted by a gain control factor generated 
by the common control 21. The gain control factor is a function 



circuits 30 (m) (1), 30 (m) (2) will normalize the respective outputs 
in relation to the highest amplitude output for their respective 
frequency. This allows each transducer pair 15 (m) and their 
20 respective CGA circuit 30 (m) to output differing values, which 
represent the color at each transducer pair 15 (m) as well as the 
"color temperature" of the light incident on the object or retina 
15. The common control 21 senses all transducer outputs for each 
frequency and uses the highest outputs 15(H) (1), 15(H) (2) to set 
each CGA circuit 3 0 (m) in the color processor 12 to the same gain 
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1 as the CGA circuits 30(H)(1), 30(H)(2) from the pixel(s) 15 (m) 

2 that sensed the highest light energy in retina 15. The 

3 transducers 15(H)(1), 15(H)(2), the CGA circuits 30(H)(1), 

4 30(H) (2) and the common control 21 operate as an automatic gain 

5 controlled loop normalizing the output signal at CGA circuit 

6 30(H) (1) . Therefore, the response of each transducer 15 (m) (1) is 

7 normalized at the output of each CGA circuit 30 (m) (1) relative to 

8 the output of CGA circuit 30(H) (1) . It is to be noted that the 

9 transducers 15(H) (1), 15(H) (2) need not be from the same pixel 

10 15 (m) , as the spectral light energy of a visual scene image at 

11 two separate frequencies is generally hot the same everywhere on 

12 retina 15. 

13 The gain controlled output of each CGA circuit 30 (m) (1), 

14 3 0 (m) (2) is provided to a number of elements, including a 

15 respective sum circuit 33 (m) , a difference circuit 32 (m) and the 
jii 16 common control 21. The outputs from the CGA circuits 30 (m) (1), 
r\ 17 3 0 (m) (2) are coupled to the difference circuit, or difference 

life 

18 generator 32 (m) , which generates an output vector that is 

19 representative of the difference between the amplitudes of the 

20 outputs form the CGA circuits 30 (m) (1), 30 (m) (2) . Accordingly, 

21 it will be appreciated that the output generated by the 

22 difference generator 32 (m) corresponds to the ratio of the 

23 amplitudes of the automatic controlled gain signals from the 

24 respective transducers 15(H) (1), 15(H) (2) and the respective 

25 pixel transducer 15 (m) outputs. 
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1 



As noted above, the outputs from the CGA circuits 30 (m) (1) 



2 and 30 (m) (2) are also coupled to a sum circuit 33 (m) . The sum 

3 circuit 33 (m) generates an output that corresponds to the sum of 

4 the amplitudes of the automatic controlled gain signal from the 

5 respective transducers 15(m)(l) and 15 (m) (2), and thus represents 

6 the brightness of the light incident on the pixel defined by the 

7 transducers . 

8 The output vector from difference circuit 32 (m) is coupled 

9 to the color boundary processor 13 (FIG. 1) . The difference 

10 vector from difference circuit 32 (m) and the brightness vector 

11 from sum circuit 33 (m) are also both coupled to a neural director 

12 35 (m) that disperses these inputs into a local color feature 

13 vector. The neural director 35 (m) is preferably similar to the 

14 neural directors as described in NEURAL DIRECTOR, U.S. Patent 

15 Application Ser. No. 09/436,957. Neural director 35 (m) is 

16 preferably established to provide an output vector with an 

17 increased dimensionality, which will aid in distinguishing 

18 between similar patterns in the input vector. 

19 The output of the neural director 35 (m) is coupled to bi- 

20 polar MKOM 36 (m) , which is described in detail in STATIC MEMORY 

21 PROCESSOR, U.S. Patent Application Ser. No. 09/477,638. The bi- 

22 polar MKOM 36 (m) generates a number of positive and/or negative 

23 outputs M(l) through M(R) , generally identified by reference 

24 numeral M(r), each of which is associated with one dimension of 

25 the feature vector input thereto. Each positive component M(r) 
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25 



of the output vector can have a range of values from zero up to a 
maximum value, which corresponds to, or is proportional to, the 
maximum positive element value of the input vector. The positive 
outputs M(r) that are associated with an input vector component 
having successively lower positive values, are themselves 
successively lower in value, thus forming a positive ranking of 
the vector components. Outputs M(r) that are associated with 
input vector components having negative values are also ranked as 
negative vector components in a similar manner to the positive 
10 components. The rankings for the respective input feature 

vectors may be global, for all of the components of the input 
feature vector, or they may be localized among a selected number 
13 of preferably contiguous input feature vector components . The 

feature vector generated by the bi -polar MKOM 3 6 (m) is coupled to 
the memory processing section 14. 

The outputs from CGA circuits 30 (m) (1) and 30 (m) (2) of all 
17 of the pixel color processors 20 (m) are also coupled to the 

common control 21. The common control 21 includes peak sensing 
circuits 40(1), 40(2), each of which receives the output from the 
correspondingly- indexed CGA circuits 30 (m) (1), 30 (m) (2), and each 
generates an output which corresponds to the one of the outputs 
from the correspondingly- indexed CGA circuits 3 0 (m) (1), 3 0 (m) (2) 
with the largest signal value. The outputs from the peak 
circuits 40(1), 40(2) are also connected to control the gain of 



-22- 






1 


all of the correspondingly- indexed CGA circuits 30(m)(l), 




2 


30(m) (2) . 




3 


The outputs from the CGA circuits 30 (m) (1) and 3 0 (m) (2) of 




4 


all of the color pixel processors 20 (m) are also connected to a 




5 


sum circuit 41. The sum circuit 41 generates an output, which 




6 


represents the sum of the outputs from all of the CGA circuits 




7 
8 


30 (m) (1), 30 (m) (2) of all of the color pixel processors 20 (m) . 
The output provided by the sum circuit 41 represents the total 




9 


intensity or power of the light incident on the retina 15. An 




10 


iris control circuit 42 uses the sum circuit 41 output to control 


if: 


11 


the iris 17, which normalizes the intensity of the light on 


i s : 
i > i 


12 


retina 15. 


ip 
IIJ 


13 


FIG. 3 is a functional block diagram of the color boundary 


\^ 


14 


processor 13, which is useful in the color sensor depicted in 




15 


FIG. 1. The color boundary processor 13 can sense a colored 




16 


object shape in a background of a different color. A black and 


7^ 


17 


white sensing retina often responds to different colors as equal 




18 


lightness. Therefore, it may not sense an object of one color 




19 


against a different background color. As noted above, the color 




20 


boundary processor 13 receives the color vector signals from the 




21 


difference circuits 32 (m) of all of the pixel color processors 




22 


2 0 (m) . Color boundary processor 13 then generates an output for 




23 


each pixel 15 (m) that represents a color gradient for the pixel 




24 


15 (m) . The outputs of each difference circuit 32 (m) are 




25 


spatially arranged exactly in the same spatial orientation as 
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1 each associated pixel 15 (m) in retina 15. The array of 

2 difference circuit 32 (m) outputs becomes a virtual retina 55, 

3 shown in FIG. 3 to aid in the visualization of the spatial 

4 interconnections between the array of color processors 20 and 

5 color boundary processor 13 . The color boundary processor 13 

6 comprises a plurality of window difference networks 50(1) through 

7 50 (M) , generally identified by reference numeral 50 (m) , each 

8 associated with one of the pixels 15 (m) and associated window 

9 57 (m) . Color boundary processor 13 further comprises a like 

10 plurality of neural directors 51 (m). 

11 Each window difference network 50 (m) receives a local window 

12 array 57 (m) of difference vectors generated by the 

13 correspondingly- indexed pixel color processor 20 (m) . Each window 

14 difference network 50 (m) , in turn, generates an output vector 

15 which represents a color acceleration vector between the 

16 difference vectors provided by the correspondingly- indexed pixel 

17 color processor 20 (m) and color vectors for pixels within a 

18 predetermined area around the pixel 15 (m) , .illustrated in FIG. 3 

19 as . local window 57 (m) . Local window 57 (m) may consist of any 

20 chosen pattern of pixels surrounding pixel 15 (m) . e.g., a star 

21 pattern or a- box pattern. Each neural director 51 (m) receives 

22 the color acceleration vector from the correspondingly- indexed 

23 window difference network 50 (m) . As with neural director 35 (m), 

24 each neural director 51 (m) is preferably established to provide 

25 an output local color boundary feature vector with the same or an 
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1 


increased dimensionality, which will aid in distinguishing 




2 


between similar patterns in the input vector. 




3 


In a modification to the invention 10, each pixel can be a 




4 


three transducer set 15 (m) . Each transducer of the set 15 (m) is 




5 


to be matched to the response of the human retinal color cones. 




6 


The three transducer set 15 (m) will produce two "transducer 




7 


pairs" for each pixel 15 (m) and with two color processors 12 a 




8 


color retina will be produced. The retina and two parallel memory- 




9 


processors 16 will sense color matched to the human color 




10 


perception over a wide range of ambient lighting conditions. 


ill 


11 


With reference again to FIG. 1, the local color feature 


111 
iij 


12 


vectors generated by the pixel color processing section 12, an 


il ! 
1 y 


13 


array of color comparators 19 and the local color boundary 


\A 

t 


14 


feature vectors generated by color boundary processor 13 for all 


; =^ 


15 


of the pixels 15 (m) , are coupled to the memory processing section 




16 


14 . The memory processing section 14 may perform a variety of 


I 


17 


individual or combined operations in connection with the feature 




18 


vectors input thereto, including object recognition and the like. 




19 


based on preselected object classification patterns or the like. 




20 


The invention provides a number of advantages . In 




21 


particular, the invention provides a system for receiving an 




22 


imacTP of an nhTe^ct and cre^neratp*? for an arrav of iDixels of the 




23 


image, color and color gradient /boundary information, in the form 




24 


of feature vectors, which may be processed to, for example. 




25 


classify the object into one of a plurality of object classes. 
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1 


The system generates the color and color gradient/boundary 




2 


information using only two transducers for each pixel, in 




3 


accordance with a bi- chromatic color recognition scheme, with the 




4 


transducers having peak responses at selected colors 1 and 2, 




5 


and a known output profile as a function of color, instead of the 




6 


non-color constancy process produced in accordance with the tri- 




7 


chromatic color recognition scheme. 




8 


It will be appreciated that numerous modifications may be 




9 


made to the system 10. For example, the memory processing 




10 


section 14 may perform processing in connection with comparisons 




11 


generated for two images, using output color feature vectors 


111 
l*J 


12 


generated either by the same color sensor 10 at two points in 


. s'h 

ilJ 


13 


time, or output comparator vectors which are generated by two 


\^ 


14 


color sensors (the second being denoted by 11' and 12') for 


i 

ill 


15 


respective pixels 15 (m) for respective images. In that case, and 




16 


with reference to FIG. 2, the color processing section 12, in 


ill 


17 


particular the pixel color processors 20 (m), may provide outputs 


\A 


18 


for the two images to the respective difference circuits 60 (m). 




19 


61 (m) of color comparison processor 19, each of which generates a 




20 


difference vector representing the difference between the 




21 


difference vectors and brightness vectors generated by the color 




22 


processors 12 for the respective images. The difference vectors 




23 


of 60 (m) and 61 (m) are input to comparator feature fusion network 




24 


array 62, which operates in a manner similar to feature fusion 




25 


network array 23. Similar difference circuits (not shown) may 
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1 also be provided for the local color boundary feature vectors 



2 generated by the color difference processors 13 for the 



3 respective images. 



4 



In addition, the peak detector circuits 40(1), 40(2) of the 



5 common control 21 may be replaced with summing circuits that 

6 generate a sum output for controlling the CGA circuits 30 (m) (1) , 

7 30 (m) (2) . 



9 adjust the iris in response to changes in the light intensity 

10 levels incident on the retina 15, so as to maintain the light 

11 levels incident on the transducers within a predetermined 

12 operating range. In that case, the CGA circuits 30 (m) (1), 

13 30 (m) (2) may have a relatively slower response to changes in the 

14 automatic gain control signals from the control circuit 21. 

15 These differences in response will allow the slower response of 

16 normalization via the CGA circuits to maintain a steady color 

17 constancy in a scene of rapid brightness changes. 

18 The described components of invention 10 provide the 

19 necessary components for a uniquely designed photographer's 

20 exposure and color temperature meter. A calibration of the 

21 common control network 21 provides values for exposure and color 

22 temperature, data. The meter may be an independent device, i.e., 

23 a hand held meter, or it may be integrated in a camera body, 

24 either electronic or film, to provide automatic exposure and 
25 



8 



Preferably, the iris control 42 will generally rapidly 
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color temperature corrections. The device may also be integrated 
into color printers or printing presses as a color ink control . 

It will be apparent that variations and modifications may be 
made to the invention herein described and illustrated, by those 
skilled in the art with the attainment of some or all of the 
advantages of the invention. It is also understood that the 
color sensor described herein may be connected to the various 
devices described in the referenced patent applications, wherein 
all the devices act in concert in a manner similar to the human 
eye. Therefore, it is the object of the appended claims to cover 
all such variations and modifications as come within the true 
spirit and scope of the invention. 



